Periphery monitoring device

ABSTRACT

A periphery monitoring device includes a coupling determiner that determines whether a towed vehicle is coupled to a towing vehicle to which the towed vehicle can be coupled; a target setter that sets a target moving position to be a target for moving at least the towed vehicle coupled to the towing vehicle; a storing controller that stores, as a moving target image, an image, including the target moving position, of a peripheral image generated by as imager provided at the towing vehicle; and an image controller that displays the stored moving target image in association with the towing vehicle or the towed vehicle included in a current image generated by the imager and currently displayed on a display device.

TECHNICAL FIELD

Embodiments of the present invention relate generally to a peripherymonitoring device.

BACKGROUND ART

Conventionally, towed vehicles (trailers) are known. A towed vehicle isrotatably coupled to the rear of a towing vehicle (tractor) for towing.The driver of a towing vehicle, while not coupled to the towed vehicle,can check rearward by viewing side-view mirrors or an image, generatedby an imager (camera) located at the rear of the towing vehicle, on adisplay near a driver's seat. However, while the towing vehicle iscoupled to the towed vehicle, the towed vehicle may partially orentirely block the viewing area of the side-view mirrors and the imagingarea of the imager depending on the coupling angle of the towed vehicle,creating blind spots. In view of this, rearview devices are proposed. Arearview device allows the driver to understand the situation behind thetowed vehicle irrespective of the coupling angle of the towed vehiclesimilarly to viewing rearward on the side-view mirrors, by displaying,on the display of the towing vehicle, images from imagers newly attachedto the lateral sides of the towed vehicle.

CITATION LIST Patent Literature

Patent Document 1: Japanese Patent No. 3436436

SUMMARY OF INVENTION Problem to be Solved by the Invention

Conventionally, however, it is necessary to attach imagers to everytowed vehicle, and provide transmission wires through which imagesgenerated by the towed vehicle are transmitted to the towing vehicle aswell as a connector device that can connect and disconnect thetransmission lines depending on whether or not the towed vehicle iscoupled. This results in cost increase. Further, the towing vehicle maybe coupled to various towed vehicles having different specifications.This makes it necessary to standardize the specifications of the displaysystem of the towing vehicle and the imaging system and the transmissionsystem of the towed vehicle, which may make it difficult to introducethe systems.

An object of the present invention is to provide a periphery monitoringdevice which can allow the driver to easily understand the surroundings,in particular, the area behind the towed vehicle without cost increase,irrespective of the specifications of the towed vehicle.

Means for Solving Problem

According to one embodiment of the present invention, a peripherymonitoring device includes a coupling determiner that determines whethera towed vehicle is coupled to a towing vehicle to which the towedvehicle can be coupled; a target setter that sets a target movingposition to be a target for moving at least the towed vehicle coupled tothe towing vehicle; a storing controller that stores, as a moving targetimage, an image, including the target moving position, of a peripheralimage generated by an imager provided at the towing vehicle; and animage controller that displays the stored moving target image inassociation with the towing vehicle or the towed vehicle included in acurrent image generated by the imager and currently displayed on adisplay device. As configured above, the periphery monitoring device candisplay the stored moving target image on the current image inassociation with the towing vehicle or the towed vehicle for example.Thus, the periphery monitoring device can indicate the relativerelationship between the target moving position and the towing vehicleor the towed vehicle by the stored moving target image irrespective ofthe contents of the current image. That is, the periphery monitoringdevice of a simple configuration can allow the driver to understand aperipheral situation.

According to one embodiment, the storing controller of the peripherymonitoring device may store at least the moving target image generatedwhen the target moving position is set. As configured above, theperiphery monitoring device can store the moving target image when thetowed vehicle is sure to move, for example. As a result, the peripherymonitoring device can ensure storing of a highly usable image whileavoiding decrease in the storage capacity of the storage due to storingof unnecessary images.

According to one embodiment, the storing controller of the peripherymonitoring device may start storing the moving target image when thetarget moving position is set. As configured above, the peripherymonitoring device can store two or more moving target images from whenthe towed vehicle is sure to move, for example. As a result, theperiphery monitoring device can store two or more moving target imagesincluding the target moving position and select a moving target imagerepresenting the relative relationship between the target movingposition and the towing vehicle or the towed vehicle appropriately fromamong the two or more images while avoiding decrease in the storagecapacity of the storage. As a result, the periphery monitoring devicecan allow the driver to more appropriately understand a peripheralsituation from the stored moving target image.

According to one embodiment, the periphery monitoring device furtherincludes a blind-spot determiner that determines whether the targetmoving position enters a dead area caused by the towed vehicle in animaging area of the imager. The image controller may display the storedmoving target image in association with the towing vehicle or the towedvehicle when the target moving position is to enter the dead area. Asconfigured above, if there is a blind spot in the imaging range causedby a turning (coupling angle) of the towed vehicle hiding the targetmoving position, the periphery monitoring device can complement the deadarea by the stored moving target image. As a result, the peripherymonitoring device can continuously display the target moving positionand maintain visibility thereof.

According to one embodiment, the image controller of the peripherymonitoring device may superimpose at least an image of the target movingposition included in the stored moving target image, on at least an areaof the dead area, when the target moving position is in the dead area onthe current image, the area corresponding to the target moving position.As configured above, if there is a dead area on the current image, forexample, the periphery monitoring device can superimpose the storedmoving target image to cover the dead area, and thereby generate animage with less strangeness representing the dead area as if no deadarea is present, and improve the visibility of the displayed image.

According to one embodiment, the image controller of the per monitoringdevice may superimpose the moving target image on the dead area in atransparent mode. As configured above, in response to a change in thesurroundings of the target moving position after storing the movingtarget image such as when a pedestrian enters, for example, theperiphery monitoring device can superimpose the moving target image in alight display mode while displaying the pedestrian in the current image.As a result, the periphery monitoring device enables the driver toeasily understand the current situation and the surroundings of thetarget moving position. Further, it is made easier for the driver torecognize presence of the dead area and the fact that the dead area iscomplemented by the moving target image, leading to alerting the driver.

According to one embodiment, when a first moving target image and asecond moving target image exhibit a difference in content equal to orgreater than a given value, the image controller of the peripherymonitoring device may display the second moving target image inassociation with the current image, the first moving target image beingstored when the target moving position is set, the second moving targetimage being stored immediately before the target moving position entersthe dead area. As configured above, with the difference being equal toor greater than a given value between the contents of the first parkingtarget image and of the second parking target image, occurrence ofchange such as motion of a movable object around the target parkingposition in a period from storing the first parking target image tostoring the second parking target image can be inferred. For example,occurrence of a change such as an entry of a pedestrian or anothervehicle parking in an area adjacent to the target moving position can beinferred. In this case, the periphery monitoring device can display thesecond moving target image reflecting the latest situation as the imageto be associated with the current image. Meanwhile, with the differencebeing less than a given value between the contents of the first parkingtarget image and of the second parking target image, no substantialchange in the surroundings of the target parking position in a periodfrom storing the first parking target image to storing the secondparking target image can be inferred. In this case, the peripherymonitoring device uses an image (first moving target image) of thetarget moving position initially recognized by the driver, such as ahigh-quality image of the target moving position generated from thefront by a lateral imager in a close position, as the image to beassociated with the current image. As a result, the periphery monitoringdevice enables the driver to easily recognize the target moving positionon the image on display.

According to one embodiment, the periphery monitoring device furtherinclude a position acquirer that acquires a current position of thetowing vehicle with reference to a position of the towing vehicle at thetime of setting the target moving position; an angle acquirer thatacquires a coupling angle between the towing vehicle and the towedvehicle; and an index acquirer that acquires a trailer indexcorresponding to a size of the towed vehicle, the trailer index beingsuperimposable on the current image. In displaying the stored movingtarget image in association with the current image, the image controllermay determine a display posture of the trailer index in accordance withthe current position of the towing vehicle and the coupling angle, anddisplays the trailer index on the current image in a superimposedmanner. As configured above, the periphery monitoring device enables thedriver to check the posture of the towed vehicle such as the turningdirection or angle on the current image, and to accurately recognize thepositional relationship between the towed vehicle and an object(obstacle or pedestrian), if found around the towed vehicle.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a side view schematically illustrating an exemplary couplingstate between a towing vehicle equipped with a periphery monitoringdevice according to an embodiment and a towed vehicle;

FIG. 2 is a top view schematically illustrating an exemplary couplingstate between a towing vehicle equipped with a periphery monitoringdevice according to an embodiment and a towed vehicle;

FIG. 3 is an exemplary configuration block diagram of a peripherymonitoring system including a periphery monitoring device according toan embodiment;

FIG. 4 is a configuration block diagram of an exemplary CPU of aperiphery monitoring device according to an embodiment;

FIG. 5 is a schematic diagram illustrating an example of setting atarget parking position (target moving position) and a guidance route bya periphery monitoring system including a periphery monitoring deviceaccording to an embodiment;

FIG. 6 is a schematic diagram illustrating an example that the targetparking position is in a dead area due to the coupling between a towingvehicle and a towed vehicle;

FIG. 7 is a schematic diagram illustrating an exemplary parking targetimage (moving target image) stored at the time of setting a targetparking position in a periphery monitoring device according to anembodiment, and illustrating an exemplary trimming area of the parkingtarget image displayed on a current image;

FIG. 8 is a schematic diagram illustrating an exemplary display on adisplay device by a periphery monitoring device according to anembodiment, and illustrating an example that the target parking positionis in a dead area due to the coupling between a towing vehicle and atowed vehicle;

FIG. 9 is a schematic diagram illustrating an example of displaying astored parking target image in a display area different from a currentoverhead image in association with a towing vehicle and a towed vehiclein a periphery monitoring device according to an embodiment;

FIG. 10 is a schematic diagram illustrating an example of displaying, injuxtaposition, an overhead image on which a stored paring target imageis superimposed corresponding to a dead area in a current overhead imageand an actual current image of a towed vehicle in the peripherymonitoring device according to an embodiment;

FIG. 11 is a flowchart of a first half of exemplary display processingof a periphery monitoring device according to an embodiment; and

FIG. 12 is a flowchart of a second half of exemplary display processingof a periphery monitoring device according to an embodiment.

DESCRIPTION OF EMBODIMENTS

Hereinafter, exemplary embodiments of the present invention will bedisclosed. Elements of embodiments described below, and actions andresults (effects) attained by the elements are merely exemplary. Thepresent invention can be implemented by elements other than thosedisclosed in the following embodiments, and can attain at least one ofvarious effects and derivative effects based on the basic elements.

FIG. 1 is a schematic side views of a towing vehicle 10 including aperiphery monitoring device according to an embodiment and a towedvehicle 12 towed by the towing vehicle 10. In FIG. 1, leftward isdefined as frontward of the towing vehicle 10, and rightward is definedas rearward of the towing vehicle 10. FIG. 2 is a top view of the towingvehicle 10 and the towed vehicle 12 illustrated in FIG. 1. FIG. 3 is anexemplary block configuration diagram of a periphery monitoring system100 including the periphery monitoring device mounted on the towingvehicle 10.

The towing vehicle 10 according to the embodiment may be an automobile(internal combustion automobile) including an internal combustion(engine not illustrated) as a power source, an automobile (electricautomobile or fuel-cell automobile) including an electric motor (notillustrated) as a power source, or an automobile (hybrid automobile)including both of the internal combustion and the electric motor as apower source. The towing vehicle 10 may be a sport utility vehicle(SUV), a pickup truck including a rear deck, or a general passengervehicle. The towing vehicle 10 can incorporate various transmissions andvarious devices or units (systems, parts and components, and etc.) fordriving the internal combustion or the electric motor. The method,number, and layout of the devices related to the driving of the wheels14 (front wheels 14F and rear wheels 14R) in the towing vehicle 10 canbe variously set.

The towing vehicle 10 includes a towing device (hitch) 18 on a rearbumper 16. The towing device 18 projects from, for example, a bottomcenter of the rear bumper 16 in a vehicle lateral direction and fixed tothe frame of the towing vehicle 10. The towing device 18 includes, forexample, a hitch ball vertically (top to bottom of the vehicle) standingand having a spherical distal end. The hitch ball is covered with acoupler located at the distal end of a coupling member 20 fixed to thetowed vehicle 12. As structured above, the towing vehicle 10 and thetowed vehicle 12 are coupled to each other while the towed vehicle 12can swing (turn) with respect to the towing vehicle 10 in the vehiclelateral direction. That is, the hitch ball of the towing device 18serves to transfer forward, backward, leftward, and rightward movementsto the towed vehicle 12 (coupling member 20), and receive accelerationor deceleration power.

The towed vehicle 12 is, for example, a box-type vehicle including atleast one of a riding space, a living area, and an accommodation space,or may be of a deck type on which luggage (e.g., a container or a boat)is to be loaded. FIG. 1 illustrates the towed vehicle 12 as a drivenvehicle including a pair of trailer wheels 22 as driven wheels and nodrive wheels and no steering wheel.

As illustrated in FIGS. 1 and 2, the towing vehicle 10 includes aplurality of imagers 24, for example, four imagers 24 a to 24 d. Theimagers 24 are, for example, digital cameras including a built-in imagesensor such as a charge coupled device (CCD) image sensor or acomplementary metal oxide semiconductor (CMOS) image sensor. The imagers24 output moving data (image data) at a given frame rate. Each of theimagers 24 includes a wide-angle lens or a fisheye lens, and can image ahorizontal range of, for example, 140 to 220 degrees. The optical axisof each imager 24 may be set obliquely downward. Thus, the imagers 24sequentially generate images of the surrounding environment outside thetowing vehicle 10 including an object (obstacle such as a pedestrian ora vehicle) and a road surface on which the towing vehicle 10 is movable,and outputs the images as image data.

The imager 24 a (rear imager) is located, for example, on the rear wallof the towing vehicle 10 below a rear hatch 10 a. The imager 24 a canimage an area (for example, the range indicated by a two--dot chain linein FIG. 1) including the rear end (rear bumper 16) of the towing vehicle10, the towing device 18, the coupling member 20, and at least the frontend of the towed vehicle 12, and an area behind the towed vehicle 12laterally viewed from the towed vehicle 12. The image data generated bythe imager 24 a can be used in recognizing the towed vehicle 12 anddetecting the coupling state (for example, coupling angle or coupling ornon-coupling) between the towing vehicle 10 and the towed vehicle 12. Inthis case, the periphery monitoring system 100 can acquire the couplingstate and the coupling angle between the towing vehicle 10 and the towedvehicle 12 from the image data generated by the imager 24 a, therefore,it can be simplified in configuration and reduce computational load orimage processing load.

The imager 24 b (left-side imager) is located, for example, at the leftend. of the towing vehicle 10, such as on a left side mirror 10 b, togenerate a leftward image including an area around the left side of thetowing vehicle 10 (for example, an area from left front to left rear).The imager 24 c (front imager) is located, for example, at the front,that is, the front end of the towing vehicle 10 in a vehicle front-reardirection, for example, on a front grill 10 c or a front bumper, togenerate a frontward image including an area ahead of the towing vehicle10. The imager 24 d (right-side imager) is located, for example, at theright end of the towing vehicle 10, for example, on a right side mirror10 d, to generate a rightward image including an area around the rightside of the towing vehicle 10 (for example, an area from right front toright rear). The periphery monitoring system 100 can execute computationand image processing to the image data generated by the imagers 24 togenerate an image with a wider viewing angle or a virtual overhead image(planar view) of the towing vehicle 10 viewed from above.

As illustrated in FIG. 3, the interior of the towing vehicle 10 isequipped with a display device 26 and an audio output device 28. Thedisplay device 26 is, for example, a liquid crystal display (LCD) or anorganic electroluminescent display (OELD). The audio output device 28is, for example, a speaker. The display device 26 is covered with, forexample, a transparent operational input 30 such as a touch panel. Theoccupant (for example, driver) can view images on the screen of thedisplay device 26 through the operation input 30. The occupant can alsotouch, press, and move the operation input with his or her finger orfingers at positions corresponding to the images displayed on the screenof the display device for executing operational inputs. The displaydevice 26, the audio output device 28, and the operational input 30 areincluded, for example, in a monitor device 32 located at a lateral orhorizontal center of a dashboard of the towing vehicle 10. The monitordevice 32 can include an operational input (not illustrated) such as aswitch, a dial, a joystick, and a push button. The monitor device 32 candouble as, for example, a navigation system or an audio system.

As illustrated in FIGS. 1 and 2, the towing vehicle 10 is, for example,a four-wheeled vehicle with two right and left front wheels 14F and tworight and left rear wheels 14R. The four wheels 14 are steerable. Asillustrated in FIG. 3, the towing vehicle 10 includes a steering system34 that steers at least two of the wheels 14. The steering system 34includes an actuator 34 a and a torque sensor 34 b. The steering system34 is electrically controlled by an electronic control unit (ECU) 36 tooperate the actuator 34 a. The steering system 34 represents, forexample, an electric power steering system, or a steer by wire (SBW)system. The steering system 34 allows the actuator 34 a to add torque,i.e., assist torque to the steering wheel to supplement the steeringforce, or turn the wheels 14. In this case, the actuator 34 a may turnone or two or more of the wheels 14. The torque sensor 34 b detects, forexample, a torque applied to the steering wheel by the driver.

As illustrated in FIG. 3, in the periphery monitoring system 100(periphery monitoring device), a brake system 38, a steering anglesensor 40, an accelerator sensor 42, a shift sensor 44, and a wheelspeed sensor 46 are electrically connected. to one another via anin-vehicle network 48 as an electric communication line in addition tothe ECU 36, the monitor device 32, and the steering system 34. Thein-vehicle network 48 is configured as, for example, a controller areanetwork (CAN). The ECU 36 can control the steering system 34 and thebrake system 38 by transmitting a control signal thereto through thein-vehicle network 48. The ECU 36 can receive results of detection fromthe torque sensor 34 b, a brake sensor 38 b, the steering angle sensor40, the accelerator sensor 42, the shift sensor 44, and the wheel speedsensor 46, and an operation signal from the operational input 30 throughthe in-vehicle network 48.

The ECU 36 includes, for example, a central processing unit (CPU) 36 a,a read only memory (ROM) 36 b, a random access memory (RAM) 36 c, adisplay controller 36 d, an audio controller 36 e, and a solid statedrive (SSD) 36 f (flash memory). The CPU 36 a reads a stored (installed)program from a nonvolatile storage such as the ROM 36 b, and executescomputation according to the program. The CPU 36 a executes, forexample, image processing to an image displayed on the display device26. For example, the CPU 36 a executes computation and image processingto the image data generated by the imagers 24 to generate a peripheralimage (for example, overhead image).

The RAM 36 c temporarily stores various kinds of data for use incalculation by the CPU 36 a. The display controller 36 d mainlysynthesizes image data displayed on the display device 26 among thecomputation of the ECU 36. The audio controller 36 e mainly processesaudio data output from the audio output device 28 among the computationof the ECU 36. The SSD 36 f is a rewritable nonvolatile storage, and canstore data after power-off of the ECU 36. The CPU 36 a, the ROM 36 b,and the RAM 36 c can. be integrated in the same package. The ECU 36 mayinclude another logical operation processor such as a digital signalprocessor (DSP) or a logical circuit, instead of the CPU 36 a, or mayinclude a hard disk drive (HDD) instead of the SSD 36 f. The SSD 36 f orthe HDD may be separated from the ECU 36.

The brake system 38 represents, for example, an anti-lock brake system(ABS) that prevents locking of the brake, an anti-skid system(electronic stability control (ESC) that prevents the towing vehicle 10from skidding during cornering, an electric brake system that increasesbraking force (executes a brake assist), or brake by wire (BBW). Thebrake system 38 applies braking force to the wheels 14 and to the towingvehicle 10 via an actuator 38 a. The brake system 38 can execute variouscontrols by detecting locking of the brake, idling of the wheels 14, andindication of skidding from a difference in rotation between the rightand left wheels 14. The brake sensor 38 b serves to sense the positionof a movable part of a brake pedal, for example.

The steering angle sensor 40 serves to, for example, detect the steeringamount of the steering wheel. The ECU 36 acquires the driver's steeringamount of the steering wheel and the steering amount of each wheel 14during automatic steering from the steering angle sensor 40 for variouscontrols. The accelerator sensor 42 serves to, for example, detect theposition of a movable part of an accelerator pedal. The shift sensor 44serves to, for example, detect the position of a movable part of a shiftoperator. The wheel speed sensor 46 is a sensor that detects therotating speed per unit time or the rotation amount of the wheels 14.The wheel speed sensor 46 outputs a wheel-speed pulse numberrepresenting the detected rotating speed as a sensor value. The ECU 36acquires a sensor value from the wheel speed sensor 46 to calculate amoving amount of the towing vehicle 10 from the sensor value for variouscontrols.

The configurations, arrangement, and electrical connection of thevarious sensors and actuators described above are merely exemplary, andcan be variously set (changed).

In moving the towed vehicle 12 coupled to the towing vehicle 10 to a settarget moving position, for example, the display device 26 can display apre-stored moving target image including the target moving position, attiming different from timing at which the moving target image is stored.For example, while the towing vehicle 10 moves toward a set targetmoving position, the towed vehicle 12, which is rotatable with respectto the towing vehicle 10, may move in a direction different from amoving direction of the towing vehicle 10 to block the target movingposition displayed in the current image on the display device 26 (hasentered a dead area). In such a case, the display device 26 displays thetarget moving position in the dead area using the stored moving targetimage. That is, the display device 26 complements the image. This makesit possible for the driver to properly, easily check the set targetmoving position while moving the towing vehicle 10 coupled to the towedvehicle 12, contributing to reducing driving load.

The following will describe parking the towed vehicle 12 in a givenparking space as an example of peripheral monitoring during moving thetowing vehicle 10 coupled to the towed vehicle 12. Thus, in thefollowing, the target moving position refers to a target parkingposition, and the moving target image refers to a parking target image.To move the towed vehicle 12, the driver may first recognize theentrance of the parking space and set the entrance as the target parkingposition. Thus, the present embodiment will describe, as an exemplarytarget parking position T, an area defined by a pair ofthree-dimensional objects such as pylons 58 a and 58 b that define thewidth of the entrance of a parking space P, as illustrated in FIG. 5. Inthis case, unless the driver sees at least one of the pylon 58 a and thepylon 58 b having entered the dead area, the driver cannot recognize thetarget parking position T since the width of the target parking positionT is indefinite. Thus, in the present embodiment, the parking targetimage refers to an image representing both the pylons 58 a and 58 bdefining the target parking position T, and containing an entrance areaof the parking space P defined by the pylons 58 a and 58 b.

The target parking position I is not limited to an area having a widthas described above, and may be set to, for example, a pinpoint positionsuch as the pylon 58 a or the pylon 58 b defining the width of theparking space P, a sectional line 60 a or a sectional line 60 h, or awidth center of the entrance of the parking space P. The presentembodiment describes, as a parking form, an example that the towingvehicle 10 moves the towed vehicle 12 to the parking space P in a givenposture (for example, substantially parallel to the sectional lines 60 aand 60 b) and detaches the towed vehicle 12 to park only the towedvehicle 12 in the parking space P.

The CPU 36 a in the ECU 36 includes various modules for executingdisplay processing to implement peripheral monitoring using thepre-stored parking target image (previous image) as described above. Thevarious modules are implemented by the CPU 36 a's reading and executingan installed and stored program from the storage such as the ROM 36 b.As illustrated in FIG. 4, for example, the CPU 36 a includes modulessuch as an acquirer 50, a target setter 52, a determiner 54, and acontrol unit 56.

In order to acquire various kinds of information to implement peripheralmonitoring, the acquirer 50 includes a peripheral monitoring-requestacquirer 50 a, an image acquirer 50 b, a coupling angle acquirer 50 c, avehicle position acquirer 50 d, a guidance route acquirer 50 e, atrailer-specification acquirer 50 f, and an index acquirer 50 g.

In response to a driver's peripheral monitoring request for parkingthrough the operational input 30 during driving the towing vehicle 10coupled to the towed vehicle 12, for example, the peripheralmonitoring-request acquirer 50 a receives a request signal. In anotherembodiment, in response to detection of the towing vehicle 10 coupled tothe towed vehicle 12 having entered a parking lot with a globalpositioning system (GPS), the peripheral monitoring-request acquirer 50a may consider the detection as a request for periphery monitoring forparking, and accept a request signal.

After the peripheral monitoring-request acquirer 50 a has acquired therequest signal, the image acquirer 50 b acquires image informationrequired for displaying the surroundings of at least the towing vehicle10. For example, the image acquirer 50 b acquires a plurality of itemsof image data (for example, frontward image data, leftward image data,rightward image data, rearward image data) from the imagers 24 a to 24 dthat generate the images of the perimeter of the towing vehicle 10. Theacquired images may be sequentially displayed on the display device 26as actual images without change or may be subjected to viewpointconversion to be sequentially displayed thereon in overhead mode. Theimages may be temporarily stored as previous images in the storage suchas the RAM 36 c or the SSD 36 f, and displayed together with the image(current image) on the display device 26 at later timing.

The coupling angle acquirer 50 c acquires a coupling angle θ between thetowing vehicle 10 and the towed vehicle 12, that is, an angle of thecoupling member 20 relative to the towing device 18 as a fulcrum. Thecoupling angle acquirer 50 c can acquire the coupling angle θ in variousmanners. For example, the coupling angle acquirer 50 c can detect thecoupling angle θ of the coupling member 20 with respect to the towingdevice 18 (towing vehicle 10) from an image based on image datagenerated by the imager 24 a. The coupling angle acquirer 50 c detects,for example, a straight line passing the coupler 20 a of the couplingmember 20, of a straight line extending in a front-rear direction of thetowed vehicle 12, and sets this straight line as a coupling axis M ofthe coupling member 20, as illustrated in FIG. 2. Since a vehicle axis Nof the towing vehicle 10 is known on the image generated by the imager24 a, the coupling angle θ can be detected from the vehicle axis N andthe coupling axis M. The present embodiment illustrates an example thatthe imager 24 a is located immediately above the towing device 18, thatis, coaxially with the vehicle axis N. Thus, the coupling member 20 canbe looked down from substantially directly above, which can facilitatedetection of the coupling angle θ between the vehicle axis N and thecoupling axis M. Meanwhile, the imager 24 a may not be placed directlyabove the towing device 18 due to structural factors of the towingvehicle 10 or other reasons. For example, the imager 24 a may be placedin a location horizontally separate from the center of the rear hatch 10a. In this case, the two-dimensional coordinates of the image generatedby the imager 24 a can be converted to three-dimensional coordinateswith reference to the ground height (known value based onspecifications) of the towing device 18 (hitch ball 18 a), to detect thecoupling angle θ on the basis of the vehicle axis N and the couplingaxis M.

In another embodiment, the coupling angle acquirer 50 c may detect thecoupling angle θ by analyzing an image representing a position of amarker attached to the coupling member 20 or the front wall surface ofthe towed vehicle 12. In still another embodiment, the towing device 18may include an angle sensor to detect the angle of the coupling member20, and the coupling angle acquirer 50 c may acquire the detected angleas the coupling angle θ. The coupling angle θ acquired by the couplingangle acquirer 50 c is used to determine whether the target parkingposition enters the dead area caused by the towed vehicle 12, or tocalculate a guidance route R for guiding the towed vehicle 12 to theparking space P as illustrated in FIG. 5. Further, the coupling angle isused to determine a display angle (display posture) of a trailer index(trailer icon) on display for the purpose of improving viewability ofthe towed vehicle 12 on the display device 26.

The vehicle position acquirer 50 d acquires a current positron (vehicleposition) of the towing vehicle 10 during stopping or traveling. Thevehicle position acquirer 50 d sets, for example, a coordinate system ofthe set target parking position T with the origin at the position of thetowing vehicle 10. The vehicle position acquirer 50 d can estimate theposition of the towing vehicle 10 in the above coordinate system fromthe turning radius of the towing vehicle 10 based on a steering anglefrom the steering angle sensor 40, a moving amount of the towing vehicle10 based on a speed from the wheel speed sensor 46, and a travelingdirection of the towing vehicle 10 from the shift sensor 44. In anotherembodiment, the vehicle position acquirer 50 d can estimate the vehicleposition through image recognition of an image based on the image dataacquired by the image acquirer 50 b. In this case, for example, thevehicle position acquirer 50 d can create an optical flow from the imagedata sequentially output from the imagers 24, and calculate the currentposition of the towing vehicle 10 from the optical flow. Further, thevehicle position acquirer 50 d may identify the current position with aGPS.

In the following, the vehicle position acquirer 50 d determines variouspositions from the above coordinate system having the origin at theposition of the towing vehicle 10 at the time of setting the targetparking position T, as an example. The vehicle position acquirer 50 d.determines, for example, the target parking position T and the positionof the stored parking target image for associating the parking targetimage with the current image, using the coordinates of this coordinatesystem. In addition, the vehicle position acquired by the vehicleposition acquirer 50 d is usable in recognition of a relative positionbetween the towing vehicle 10 and the target parking position T ordetermination on whether the target parking position T enters the deadarea caused by the towed vehicle 12. The vehicle position can also beused to calculate the guidance route R for guiding the towed vehicle 12to the parking space P.

As illustrated in FIG. 5, for example, the guidance route acquirer 50 eacquires the guidance route R, serving to guiding the towing vehicle 10from the current position to the target parking position T in theparking space P on, for example, a coordinate system (X-Z, coordinates)defined when the target parking position T is set. The guidance routeacquirer 50 e sets, for example, a guidance reference point Bsubstantially at the center of the axle connecting the right and leftrear wheels 14R of the towing vehicle 10, and calculates the guidanceroute R such that the guidance reference point B substantially matches atarget guidance point C set prior to the start of guidance. In thepresent embodiment, the center of the target parking position T definedby the pylons 58 a and 58 b located at the entrance of the parking spaceP is defined as the target guidance point C. By moving the guidancereference point B of the towing vehicle 10 to the target guidance pointC, the towed vehicle 12 coupled to the rear of the towing vehicle 10 canbe placed in the parking space P. The guidance route R can be found(calculated) by a known method, therefore, a detailed descriptionthereof is omitted. The guidance route R of the towing vehicle 10 iscalculated such that the towed vehicle 12 can be moved to the parkingspace P in a given posture with the minimum number of turns in ashortest distance from the guidance reference point B representing thecurrent position of the towing vehicle 10 to the target guidance pointC. Herein, the given posture refers to, for example, the state of thetowed vehicle 12 in the parking space P with lateral and longitudinalgaps relative to the parking space P being in a given range, and anangle between the longitudinal center line of the parking space P andthe towed vehicle 12 being a given value or less. The current position(guidance reference point B) of the towing vehicle 10 and the targetguidance point C may be transmitted to an external system (for example,a parking-lot management system) to calculate guidance route R. Theguidance route acquirer 50 e may acquire the guidance route R from theexternal system, for example.

The trailer-specification acquirer 50 f acquires specifications of thetowed vehicle 12 (for example, sizes of the towed vehicle 12). Asillustrated in FIG. 5, the imager 24 d located on the right lateralsurface of the towing vehicle 10 can generate an image of the targetparking position (area including the pylons 58 a and 58 b) in theparking space P when the towing vehicle 10 coupled to the towed vehicle12 passes ahead of the parking space P. As illustrated in FIG. 6, whilethe towing vehicle 10 coupled to the towed vehicle 12 is moving alongthe guidance route R (refer to FIG. 5), the coupling angle θ of thetowed vehicle 12 may change, resulting in hiding the target parkingposition T partially or entirely on the display device 26. In FIG. 6,the pylon 58 a included in the target parking position is located in adead area D caused by the towed vehicle 12, and thus cannot be seen onthe display device 26. In this case, the driver may not be able to checkone end of the target parking position T, and be difficult to know thecondition of the parking space P (size of the parking space P orpositional relationship with respect to a towed vehicle 12 a (refer toFIG. 5)) parked in an adjacent parking space P1. To determine whetherthe target parking position I (pylon 58 a) is in the dead area D, theactual size of the towed vehicle 12 is needed. Thus, thetrailer-specification acquirer 50 f acquires information such as thelongitudinal length and the width of the towed vehicle 12 input throughthe operational input 30.

In order to calculate the guidance route R for allowing the towedvehicle 12 to move to the parking space P defined by the target parkingposition T, the guidance route acquirer 50 e is to understand thebehavior of the towed vehicle 12 rotatably coupled to the towing vehicle10 via the towing device 18. During backward travel of the towingvehicle 10, the towed vehicle 12 rotatable with respect to the towingvehicle 10 may behave the same as or differently from the towing vehicle10. For example, while the towing vehicle 10 and the towed vehicle 12are coupled in a balanced state, the towing vehicle 10 and the towedvehicle 12 behave in the same manner. That is, the towing vehicle 10 andthe towed vehicle 12 can be regarded as a united object to know theirmoving state. For example, with the vehicle axis N of the towing vehicle10 and the coupling axis M of the towed vehicle 12 substantiallymatching (substantially becoming one line), the towing vehicle 10 andthe towed vehicle 12 are placed in a balanced state. With the vehicleaxis N and the coupling axis M not matching but the turning centers ofthe towing vehicle 10 and of the towed vehicle 12 substantiallymatching, the towing vehicle 10 and the towed vehicle 12 is placed in abalanced state. The turning center of the towing vehicle 10 can be foundfrom a current steering angle of the towing vehicle 10 and the length ofthe wheelbase of the towing vehicle 10. The turning center of the towedvehicle 12 can be found from the coupling angle θ between the towingvehicle 10 and the towed vehicle 12 and the length of the wheelbase ofthe towed vehicle 12. The length of the wheelbase of the towed vehicle12 is from the towing device 18 to the axle of the trailer wheel 22 ofthe towed vehicle 12 including the coupling member 20. However, thetowing vehicle 10 can be coupled to the towed vehicles 12 having variousspecifications (lengths), and the length of the wheelbase differsdepending on the specifications of the towed vehicle 12. Thus, thetrailer-specification acquirer 50 f allow the driver to directly inputthe wheelbase length of the towed vehicle 12 for determining thebalanced state through the operational input 30 together withinformation such as the longitudinal length and the width of the towedvehicle 12. The driver can input such specifications, referring to thespecification sheet of the towed vehicle 12, for example.

The index acquirer 50 g reads, from a list stored in the storage such asthe ROM 36 b, a trailer index (trailer icon) of a size and shapecorresponding to the longitudinal length and the width of the towedvehicle 12 acquired by the trailer-specification acquirer 50 f. To forman overhead image through image processing including viewpointconversion or synthesis to the images generated by the imagers 24,various kinds of correction may not be sufficient to resolve distortionand extended shape of the images. For example, the towed vehicle 12 maybe extended or deformed as further away from the viewpoint, which makesit difficult to know the positional relationship between the towedvehicle 12 and peripheral objects (for example, the pylons 58 a and 58b). In such a case, superimposing the trailer icon corresponding to theactual shape on the current image makes it easier for the driver to knowthe positional relationship between the towed vehicle 12 and peripheralobjects. For display of the overhead image, the index acquirer 50 g alsoacquires a vehicle icon representing the vehicle (towing vehicle 10)which is not displayable based on the image data generated by theimagers 24. The vehicle icon and the trailer icon can be changed incoupling posture on display in accordance with the coupling angle θacquired by the coupling angle acquirer 50 c.

In response to receipt of the request signal for peripheral monitoringby the peripheral monitoring-request acquirer 50 a, the target setter 52enables setting of the target parking position T. For example, afterdisplay of an image based on the image data acquired by the imageacquirer 50 b on the display device 26 in response to the peripheralmonitoring request, the driver can designate a position on the displaydevice 26 with the operational input 30. The driver designates aposition in an intended parking location displayed on the display device26 using the operational input 30. After the CPU 36 a determines theposition as suitable for parking the towed vehicle 12, the target setter52 sets the position as the target parking position T. Parkingsuitability of a driver's designated location relative to the towedvehicle 12 can be determined by a known method. For example, throughimage analysis of the images generated by the imagers 24, parkingsuitability can be determined from the interval between the pylon 58 aand the pylon 58 b being greater than the width of the towed vehicle 12by a given value or more, the depth of the parking space P being greaterthan the longitudinal length of the towed vehicle 12 by a given value ormore. The towing vehicle 10 may include a ranging device such as sonar,and the target setter 52 may determine whether or not to be able to setthe target parking position T in a designated space from a result of theranging. In addition, the target setter 52 may present, on the displaydevice 26, one or two or more candidates for the parking space P wherethe target parking position T can be set, and allow the driver to selecta desired parking position from among the candidates.

The determiner 54 includes a coupling determiner 54 a that determinesthe coupling state between the towing vehicle 10 and the towed vehicle12 and a blind-spot determiner 54 b that determines whether the targetparking position T is in the blind spot of the towed vehicle 12.

For example, the coupling determiner 54 a can determine the couplingstate on the basis of input information which has been input to theoperational input 30 by the driver of the towing vehicle 10 whencoupling the vehicle to the towed vehicle 12. After recognition of thetowed vehicle 12 from the image, subjected to image processing, based onthe image data representing the area behind the towing vehicle 10acquired by the image acquirer 50 b, the coupling determiner 54 a candetermine the coupling state on the basis of the recognitioninformation. The towing device 18 may include a sensor that senses thecoupling between the towing device 18 and the coupling member 20, andthe coupling determiner 54 a may determine the coupling state on thebasis of detected information. The towed vehicle 12 coupled to thetowing vehicle 10 is subjected to lighting control of stop lamps,direction indicators, and width indicators located on the rear end ofthe towed vehicle 12 under the control of the towing vehicle 10. In thiscase, control wires connect between the towing vehicle 10 and the towedvehicle 12. The coupling determiner 54 a may determine the couplingstate in accordance with a signal representing the connection of thecontrol lines.

The blind-spot determiner 54 b determines whether the target parkingposition T is hidden by the towed vehicle 12 on the image displayed onthe display device 26. As illustrated in FIG. 6, for example, theimaging range is defined by the angle of view and the posture of theimager 24 d on the right side of the towing vehicle 10 (one dot chainline). Meanwhile, the dead area D (hatched area) changes depending onthe coupling angle θ or the rowed vehicle 12 relative to the towingvehicle 10 (angle between the vehicle axis N and the coupling axis M)(two-dot chain line). Thus, the range of the dead area D is determinedfrom, for example, the coupling angle θ of the towed vehicle 12 and thelength and width of the towed vehicle 12, if found on the overhead image(on the coordinate system having the origin at the position of thetowing vehicle 10 when the target parking position T is set). Further,the blind-spot determiner 54 b can determine whether the pylon 58 a orthe pylon 58 b defining the end of the target parking position I entersthe dead area D from the relative position between the current positionand the target parking position T of the towing vehicle 10, if found onthe coordinate system. That is, the blind-spot determiner 54 b candetermine in real time whether the target parking position T enters thedead area D. In another embodiment, the blind-spot determiner 54 b cancalculate similarity between two or more images by a known imagedifference method or a correlation calculation method (such as templatematching using normalized cross-correlation (NCC)) to determine whetherthe target parking position T is in the dead area D. For example, abenchmark image including the target parking position T (for example, animage of the target parking position T when set) and a current referenceimage to contain a most current target parking position T are subjectedto viewpoint conversion for comparison. Thereby, the blind-spotdeterminer 54 b can determine whether the pylon 58 a, for example,appearing in the benchmark image, disappears in the reference imagethrough comparison between the two images. In this case, the blind-spotdeterminer 54 b can determine in real time whether the target parkingposition T enters the dead area D.

The control unit 56 includes a storing controller 62 and an imagecontroller 64. The storing controller 62 controls timing at which theimage data generated by the imagers 24 and acquired by the imageacquirer 50 b is stored. In the case of storing all the image datagenerated by the imagers 24, for example, the RAM 36 c or the SSD 36 frequires enormous storage capacity, causing cost increase and increasein processing load of the stored image data. In the present embodimentdescribed above, the parking target image including the target parkingposition T is displayed in association with the towing vehicle 10 or thetowed vehicle 12 on the current image and is used in complementing thedead area D, by way of example. Thus, after determination that thevehicle is actually to park, e.g., when the target parking position T isset, the storing controller 62 stores the image based on the image dataacquired by the image acquirer 50 b as the parking target image,together with the coordinates of the target parking position T in thecoordinate system having the origin at the position of the towingvehicle 10 when the target parking position T is set. The driver can setthe target parking position T after finding (recognizing) an areasuitable for parking, for example. In this case, the target parkingposition T is likely to be imaged at an angle or in a close distancerecognizable from the imagers 24. FIG. 7 illustrates an exemplaryparking target image (actual image RV) stored when the target parkingposition T is set. As illustrated in FIG. 7, for example, the imager 24d is likely to be able to capture the target parking position T, thepylons 58 a and 58 b, and the parking space P in the vicinity, andgenerate a high-quality image with high resolution even after subjectedto image processing for display.

To move the towed vehicle 12 to the parking space P, the towing vehicle10 typically passes by the parking space P once and then travelsbackward to the parking space P. Thus, starting storing the parkingtarget image, for example, when moving the towing vehicle 10 backwardmay result in storing a blurred image. For example, the target parkingposition T may be already located in the dead area D due to the towedvehicle 12 or may be in a long distance. Such an image may be blurred ascompared with an image generated when the parking space P (targetparking position T) is initially identified. Thus, it is preferable tostore the image generated at the time of initially identifying thetarget parking position T as the parking target image in terms ofacquiring a higher-quality image.

After the target parking position T is set and the parking target imageis stored, a pedestrian may enter the parking space P or the parkingstate of the towed vehicle 12 a. in the adjacent parking space P1 maychange, for example. In order to understand such a change in thesurroundings of the parking space P (target parking position T), thestoring controller 62 may start storing, as the parking target image,the image based on the image data acquired by the image acquirer 50 btogether with the coordinates of the target parking position T,concurrently with setting of the target parking position T. In thiscase, the storing controller 62 may sequentially store the parkingtarget image while the towing vehicle 10 is moving, and may not storethe parking target image while the towing vehicle 10 is stopping sincethe target parking positon T does not change in position with respect tothe towing vehicle 10 or the towed vehicle 12. This results incontributing to reducing the storage capacity for storing the parkingtarget image.

Since the blind-spot determiner 54 b can determine timing at which thetarget parking position T for example, the pylon 58 a or the pylon 58 b)enters the dead area D, the storing controller 62 can store the parkingtarget image of the target parking position T when set and the parkingtarget image thereof immediately before entering the dead area D in adistinctive manner. For example, the storing controller 62 can store, asa first parking target image (moving target image), the image of thetarget parking position T (target moving position) when set, togetherwith the coordinates of the target parking position T. The storingcontroller 62 can also store, as a second parking target image, theimage of the target parking position T immediately before entering thedead area ID, together with the coordinates of the target parkingposition T. Usage of the second parking target image will be describedbelow in detail.

The image controller 64 includes an image-conversion controller 64 a, asynthesis controller 64 b, and a comparison controller 64 c for variouskinds of image processing to the image based on the image data acquiredby the image acquirer 50 b and the parking target image stored by thestoring controller 62.

The image-conversion controller 64 a performs viewpoint conversion tothe image data acquired by the image acquirer 50 b to generate a virtualoverhead image TV (planar image) of the towing vehicle 10 or the towedvehicle 12 viewed from above, as illustrated in FIG. 8. The overheadimage TV depicts a trailer icon 66 a corresponding to the towed vehicle12, a vehicle icon 66 b corresponding to the towing vehicle 10, and thepylon 58 b. FIG. 8 illustrates the dead area D due to the towed vehicle12 by hatching and the pylon 58 a hidden by the dead area ID by a brokenline for the sake of convenience, however, the pylon 58 a is notactually displayed at this point. FIG. 8 illustrates an example ofdisplaying the image based on the image data currently generated by theimager 24 on the display device 26, showing in juxtaposition theoverhead image TV and the actual image RV generated by the imager 24 a,being a rearward image of the area behind the towing vehicle 10. In FIG.8, the actual image RV depicts the rear bumper 16 and the towing device18 of the towing vehicle 10 in the bottom, and the coupling member 20and the front end wall of the towed vehicle 12 in a display area abovethe rear bumper 16. In addition, the actual image RV shows the pylon 58b located outside the dead area D caused by the towed vehicle 12.

The image-conversion controller 64 a performs image conversion to theparking target image stored by the storing controller 62 for the purposeof facilitating image synthesis for display in association with thecurrent image including the dead area D. As illustrated in FIG. 7, forexample, the image-conversion controller 64 a can trim the partincluding the target parking position T from the parking target image(actual image RV) stored by the storing controller 62, perform viewpointconversion to the target parking position T to generate an imagecorresponding to the overhead image TV currently displayed on thedisplay device 26, or rotate or enlarge or shrink the target parkingposition T.

The synthesis controller 64 b displays the trailer index (trailer icon66 a) and the vehicle icon 66 b acquired by the index acquirer 50 g onthe overhead image TV in a superimposed manner, as illustrated in FIG.8. The synthesis controller 64 b changes the coupling angle between thetrailer icon 66 a and the vehicle icon 66 b on the overhead image TV inreal time so as to correspond to the current coupling angle θ betweenthe towing vehicle 10 and the towed vehicle 12 acquired by the couplingangle acquirer 50 c. As described above, in generating the overheadimage TV from the image data generated by the imagers 24, for example,the image of the towed vehicle 12 may be extended long rearward. In sucha case, display of the trailer icon 66 a can help the driver understandthe posture and shape of the towed vehicle 12, the coupling state withrespect to the towing vehicle 10, and the relationship between thesurroundings and the towed vehicle 12 on the overhead image TV.

As illustrated in FIG. 8, for example, after the blind-spot determiner54 b determines that the pylon 58 a defining the target parking positionT is in the dead area D caused by the towed vehicle 12 (the targetparking position T is hidden), the synthesis controller 64 b cancomplement the dead area D to generate a synthetic image which presentsthe target parking position T in a viewable manner, as illustrated inFIGS. 9 and 10.

For example, FIG. 9 illustrates an example of displaying an enlargedview of near the dead area D hiding part of the target parking positionT on the overhead image TV on the display device 26, instead of theactual image RV in FIG. 8. In the coordinate system having the origin atthe position of the towing vehicle 10 at the time of setting the targetparking position T, the coordinates of the current position of thetowing vehicle 10 and the coordinates of the target parking position Tcan be found. The storing controller 62 stores the coordinates of thetarget parking position T together with the parking target image(properly depicting the target parking position I) of the target parkingposition T when set. Thus, after the blind-spot determiner 54 bdetermines that the target parking position T is in the dead area D, thesynthesis controller 64 b superimposes a parking target image of thetarget parking position T when set stored by the storing controller 62,having been subjected to as viewpoint conversion, trimming, rotation,and scaling by the image-conversion controller 64 a, on part of theenlarged current image displayed on the display device 26, the partcorresponding to the target parking position T. In this case, the storedparking target image is superimposed on the part of the current imagecorresponding to the target parking position T to match the coordinates,whereby a synthetic image with less strangeness can be generated.

In the display example of FIG. 9, different windows display the currentoverhead image TV of the target parking position T (pylon 58 a) hiddenby the towed vehicle 12, and a complementary overhead image TV1 injuxtaposition. In the complementary overhead image TV1 the targetparking position T (pylon 58 a) in the dead area D is viewable owing tothe stored parking target image. In this case, the driver can recognizethe dead area D hiding the target parking position T and thecomplemented part (displaying the stored parking target image) on thedisplay device 26 at the same time. In addition, the positionalrelationship between the target parking position T and the towed vehicle12 is made easily understandable. This makes it possible for the driverto more easily move the towed vehicle 12 (drive the towing vehicle 10)with a sense of safety.

FIG. 10 illustrates another example of displaying the parking targetimage stored by the storing controller 62 to complement the part hiddenby the dead area D on the overhead image TV on the display device 26 inFIG. 8. In FIG. 10, the actual image RV representing that the targetparking position T (pylon 58 a) is in the dead area D is displayed as itis. After the blind-spot determiner 54 b determines that the targetparking position T is in the dead area D, the synthesis controller 64 bsuperimposes the parking target image stored by the storing controller62, on the part of the current overhead image TV corresponding to thetarget parking position T on the display device 26. In this case, thesynthesis controller 64 b performs viewpoint conversion, trimming,rotation, and scaling to the parking target image of the target parkingposition T when set. The synthesis controller 64 b superimposes thecurrent image and the parking target image to match the coordinates. Thesynthesis controller 64 b superimposes the parking target image not onthe entire current image but uses the trimmed parking target image tocomplement the minimal part hidden by the guidance route R. This makesit possible to resolve inconvenience that the entire current imagereturns to the image representing the target parking position T whenset. In addition, the synthesis controller 64 b may change thetransmittance of the parking target image to superimpose. For example,the synthesis controller 64 b displays a synthetic image representing nodead area D by superimposing the parking target image at lowtransmittance. Conversely, at increased transmittance of the parkingtarget image to superimpose, the stored parking target image can bedisplayed in a light display mode while the current image is maintained.As a result, the synthetic image can reflect change in situation aroundthe most current target parking position T, if such a change occursafter storing of the parking target image, such as when a pedestrianenters or the parking status of the adjacent parking space P1 changes.Display of the parking target image in a transparent mode allows thedriver to easily recognize presence of the guidance route R and the factthat the guidance route R is complemented by the parking target image,contributing to alerting the driver. The driver may change the settingof transmittance with the operational input 30 when appropriate or thetransmittance may be set to a fixed value.

In the display example of FIG. 10, the overhead image TV and the actualimage RV are continuously displayed from start of the peripheralmonitoring. After the blind-spot determiner 54 b determines that thetarget parking position T is in the dead area D, the part hidden by thedead area D and its periphery are complemented by the stored parkingtarget image. This enables continuous display of peripheral informationof the target parking position T without a sudden change in the image ondisplay. Due to no significant change in image contents, the driver cancontinue to view the image without a sense of strangeness. In thedisplay example of FIG. 10, the actual image RV is continuouslydisplayed unlike display of only the converted screens with the icons inFIG. 9. Thus, display of the actual image RV can provide a sense ofsafety to the driver. The driver may select the display mode in FIG. 9or the display mode in FIG. 10, or the control unit 56 may decide thedisplay mode depending on current situation.

As described above, the synthesis controller 64 b complements thecurrent image by the parking target image, for example, triggered by thetarget parking position T entering the dead area D. This enables smoothimage display after the complementation and can immediately abateinfluence of the dead area D, if occurs. This results in reducingstrangeness of the current image on display, enabling the driver toeasily move the towed vehicle 12 (drive the towing vehicle 10) with asense of safety.

The comparison controller 64 c compares two or more parking targetimages stored by the storing controller 62, and selects a parking targetimage to be associated with the current image, to thereby reflect changein the situation around the target parking position T in the currentimage, as in the change in the transmittance. As described above, thestoring controller 62 starts storing the parking target image when thetarget parking position I is set. That is, the storing controller 62stores the change in the situation around the target parking position Iin chronological order. In view of this, the comparison controller 64 ccompares the contents of the first parking target image stored when thetarget parking position T is set with the contents of the second parkingtarget image stored immediately before the blind-spot determiner 54 bdetermines that the target parking position T is in the dead area D.Such a comparison can be implemented by a general image recognitionmethod such as normalized cross-correlation (NCC) or the sum of absolutedifferences (SAD: using the sum of absolute values of differences inluminance values).

After finding a difference as equal to or more than a given valuebetween. the first parking target image and the second parking targetimage as a result of comparison, the comparison controller 64 c selectsthe second parking target image to display in association with thecurrent image. That is, a difference being a given value or more betweenthe first parking target image and the second parking target image canbe regarded as occurrence of change in the situation around the targetparking position T before the target parking position T enters the deadarea D. A pedestrian's entry or a change in the parking status of theadjacent parking space P1, for example, can be inferred. After theblind-spot determiner 54 b determines that the target parking position Tis in the dead area D, the synthesis controller 64 b displays the secondparking target image selected by the comparison controller 64 c inassociation with the current image. As a result, the synthesiscontroller 64 b can synthesize a complementary image more accuratelyreproducing the current peripheral situation as compared withassociating the current image with the first parking target image(stored when the target parking position T is set). In addition, withoutincrease in the transmittance of the second parking target image, thesynthesis controller 64 b can generate a viewable synthetic image withless strangeness representing the target parking position T in a densemanner.

With the difference being less than a given value between the contentsof the first parking target image and of the second parking targetimage, no substantial change in the situation around the target parkingposition T from storing the first parking target image to storing thesecond parking target image can be inferred. In this case, the synthesiscontroller 64 b uses the first parking target image stored when thedriver first recognizes the target moving position as the image to beassociated with the current image. As described above, the first parkingtarget image is likely to be an image of the target parking position Tcaptured from the front in a close distance with the lateral imager 24d, for example. Thus, the synthesis controller 64 b can subject such animage to image processing including viewpoint conversion to beassociated with the current image, to synthesize the image with thecurrent image without a significant decrease in resolution. As a result,viewable synthetic images can be generated as compared with using thesecond parking target image. In this case, no substantial change in thesituation around the target parking position T can be inferred, so thatthe first parking target image may remain low in transmittance.

As described above, the second parking target image is generated whilethe towing vehicle 10 (the towed vehicle 12) is moving for parking.Thus, the second parking target image may be generated farther away fromtarget parking position than the first parking target image or thetarget parking position may be imaged in the periphery of the imagingrange. In this case, viewpoint conversion to the second parking targetimage for display associated with the current image may reduce theresolution thereof and lower the image quality than the first parkingtarget image, which is likely to be generated closely to the targetparking position T from the front. Thus, the driver can select, throughthe operational input 30, display of the first parking target image withhigher transmittance, which is likely to exhibit higher resolution andlight representation, or the second parking target image, which islikely to exhibit lower resolution and easily viewable denserepresentation, for example.

Display processing by the periphery monitoring device (peripherymonitoring system 100) configured as above will be described withreference to the flowcharts in FIGS. 11 and 12. FIG. 11 is a flowchartillustrating the first half of the processing, and FIG. 12 is aflowchart illustrating the second half of the processing. The peripherymonitoring device performs the processing illustrated in FIGS. 11 and 12at a given cycle upon ON of the ignition switch of the towing vehicle10, for example.

First, the CPU 36 a checks whether the towed vehicle 12 is coupled tothe towing vehicle 10 via the coupling determiner 54 a (S100). After thecoupling determiner 54 a fails to determine the coupling of the towedvehicle 12 (No in S100), the CPU 36 a temporarily ends the flow. Afterthe coupling determiner 54 a confirms the coupling of the towed vehicle12 (Yes in S100), the CPU 36 a checks whether the trailer-specificationacquirer 50 f has acquired the specifications of the towed vehicle 12(S102). For example, with no inputs of the specifications of the towedvehicle 12 via the operational input 30 (No in S102), thetrailer-specification acquirer 50 f displays a screen to prompt thedriver to input the specifications of the towed vehicle 12 on thedisplay device 26 and acquire the specifications of the towed vehicle 12(S104). If the trailer-specification acquirer 50 f has acquired thespecifications of the towed vehicle 12 (Yes in S102), the CPU 36 a skipsthe operation of S104.

Subsequently, the CPU 36 a checks whether the peripheralmonitoring-request acquirer 50 a has acquired a request signalrepresenting start of peripheral monitoring. With no request signalacquired (No in S106), the CPU 36 a temporarily ends the flow. After theperipheral monitoring-request acquirer 50 a acquires a request signalrepresenting start of peripheral monitoring (Yes in S106), the imageacquirer 50 b acquires image data (images) generated by the imagers 24(24 a to 24 d) (S108). The image-conversion controller 64 a performsimage processing including viewpoint conversion to the image dataacquired by the image acquirer 50 b, to generate an overhead image TV asillustrated in FIG. 8, for example. The control unit 56 switches anormal screen as a navigation screen or an audio screen on the displaydevice 26 to a peripheral monitoring screen displaying the overheadimage TV generated by the image-conversion controller 64 a and theactual image RV as rearward image generated by the imager 24 a, anddisplays a peripheral image of the towing vehicle 10 (S110). In thiscase, the index acquirer 50 g reads the vehicle icon 66 b from the ROM36 b and displays the vehicle icon 66 b at a given position on theoverhead image IV.

After start of the peripheral monitoring, the CPU 36 a checks whetherthe target setter 52 has set the target parking position T. When notarget parking position T is set (No in S112), the CPU 36 a temporarilyends the flow. After the target setter 52 sets the target parkingposition T (Yes in S112), the storing controller 62 stores an imageincluding the target parking position T set as a parking target image inassociation with the coordinates of the target parking position T(S114). The storing controller 62 starts continuously storing theparking target image including the target parking position T.

On the coordinates having the origin at the current position of thetowing vehicle 10 when the target parking position T is set, the targetparking position T is represented by the coordinates relative to theorigin. The vehicle position acquirer 50 d acquires (estimates) thecurrent position of the towing vehicle 10 on the coordinates (S116). Asillustrated in FIG. 5, the guidance route acquirer 50 e acquires(calculates) a guidance route R from the current position of the towingvehicle 10 (for example, guidance reference point B as a center of theaxle of the rear wheels 14R) to the target guidance point C setcorresponding to the target parking position T (S118). The CPU 36 aexecutes route guidance to move the guidance reference point B (towingvehicle 10) along the acquired guidance route R (S120). This routeguidance may be implemented by displaying a driver's operation such assteering, acceleration, and braking on the display device 26 oroutputting audio through the audio output device 28, thereby allowingthe driver to move the towing vehicle 10 along the guidance route R. Inanother embodiment, the CPU 36 a may automatically execute all thedriving operations including steering, acceleration, and braking usingvarious sensors or actuators mounted on the towing vehicle 10, to movethe towing vehicle 10 along the guidance route R. Alternatively, the CPU36 a may automatically execute part of the operations and provide theremaining operations to the driver through a display or audio to performthe remaining operations.

After start of the route guidance to move the towing vehicle 10 and thetowed vehicle 12, the coupling angle acquirer 50 c starts acquiring thecoupling angle between the towing vehicle 10 and the towed vehicle 12(S122). The index acquirer 50 g reads, from the ROM 36 b, the trailericon 66 a to superimpose on the overhead image of the towed vehicle 12on the overhead. image TV on the basis of the specifications of thetowed vehicle 12 acquired by the trailer-specification acquirer 50 f.The synthesis controller 64 b displays the trailer icon 66 a on theoverhead image TV in a superimposed manner (S124). The overhead image ofthe towed vehicle 12 can be displayed on the overhead image TV throughviewpoint conversion to the image data, but may be extended long inshape and be inaccurately displayed, for example. Thus, the indexacquirer 50 g may acquire the trailer icon 66 a upon display of theoverhead image TV, irrespective of start of the route guidance, and thesynthesis controller 64 b may display the trailer icon 66 a.

The blind-spot determiner 54 b determines a blind spot as to whether thetarget parking position T enters in the dead area D while the towingvehicle 10 (the towed vehicle 12) is moving (S126). If the targetparking position T is in the dead area D (Yes in S128), the comparisoncontroller 64 c compares the contents of the first parking target imageof the target parking position T when set stored by the storingcontroller 62, with the contents of the second parking target image ofthe target parking position T immediately before entering the dead areaD (S130). With the difference in content being not equal to or greaterthan a given value between the first parking target image and the secondparking target image as a result of the comparison (No in S132), thesynthesis controller 64 b selects the first parking target. image of thetarget parking position T when set (S134). Then, the synthesiscontroller 61 b displays the selected first parking target image(previous image) in association with the current image currentlydisplayed on the display device 26 (S136). That is, as in the examplesillustrated in FIGS. 9 and 10, the target parking position T hidden bythe dead area D is complemented by a relatively clear, high-qualityparking target image, which is stored at the time of setting the targetparking position T. This allows the driver to easily recognize thepositional relationship between the target parking position T and thetowing vehicle 10 and the towed vehicle 12 and a moving status of thetowed vehicle 12 with respect to the parking space P.

In S132, with the difference in content being equal to or greater than agiven value between the first parking target image and the secondparking target image as a result of the comparison (Yes in S132), thesynthesis controller 64 b selects the second parking target image of thetarget parking position T immediately before entering the dead area D,stored by the storing controller 62 (S138). The synthesis controller 64b displays the selected second parking target image (previous image) inassociation with the current image currently displayed on the displaydevice 26 (S136). That is, the synthesis controller 64 b complements thecurrent image by the parking target image reflecting the surroundings ofthe target parking position T immediately before the target parkingposition T is hidden by the dead area D. As a result, the synthesiscontroller 64 b generates a synthetic image reflecting a situation, forexample, that a pedestrian enters the target parking position T or theparking space P in a period from setting the target parking position Tto the target parking position entering the dead area D. In this case,although the second parking target image may lower in resolution, asdescribed above, the synthetic image reflecting the latest peripheralsituation can allow the driver to recognize the positional relationshipbetween the target parking position and the towing vehicle 10 and thetowed vehicle 12, and a moving status of the towed vehicle 12 withrespect to the parking space P.

In S128, if the target parking position T is not in the dead area D (Noin S128), the CPU 36 a skips the operation of S130 to S138. The CPU 36 adetermines whether the towing vehicle 10 (guidance reference point B)has reached the target parking position T (target guidance point C). Ifit has (Yes in S140), the CPU 36 a ends the peripheral monitoring(S142). That is, the control unit 56 returns the display of theperipheral monitoring screen to the normal display as a navigationscreen or an audio screen on the display device 26. If the towingvehicle 10 (guidance reference point B) has not reached the targetparking position T (target guidance point C) (No in S140), the CPU 36 aproceeds to S114 and performs the processing from S114. In this case,the CPU 36 a stores a parking target image of the towing vehicle 10 atthe current moving position and acquires the current position of thetowing vehicle 10 again, to eliminate the difference between theposition of the towing vehicle 10 and the stored image. In addition, theCPU 36 a acquires the guidance route R from the current position of thetowing vehicle 10 to the target guidance point C again for errorcorrection.

The flowcharts illustrated in FIGS. 11 and 12 are exemplary. Replacementor increase or decrease of the steps is applicable when appropriate aslong as the currently displayed image can be complemented by thepre-stored parking target image including the target parking position T,when the target parking position T is in the dead area D. Thereby, thesame or similar effects are attainable. For example, in place of S130 toS138, the parking target image, stored at the time of setting the targetparking position T, may be changed in transmittance and displayed inassociation with the current image.

As described above, the periphery monitoring device (peripherymonitoring system 100) according to the present embodiment can employ asystem including the existing imagers 24 (24 a to 24 d) of the towingvehicle 10, to provide, without cost increase, a peripheral monitoringimage which allows the driver to easily understand a peripheralsituation irrespective of the specifications of the towed vehicle 12.The periphery monitoring device provides such a peripheral monitoringimage to the driver, thereby enabling the driver to easily and safelymove the towed vehicle 12 into the parking space P.

The above embodiment has described an example of parking only the towedvehicle 12 in the parking space P. That is, the towing vehicle 10detaches the towed vehicle 12 and moves after parking the towed vehicle12. However, the towing vehicle 10 may be parked in the parking space Ptogether with the towed vehicle 12. In this case, the peripherymonitoring device determines whether the parking space P is large enoughto accommodate the towing vehicle 10 and the towed vehicle 12 at thetime of setting the parking space P or the target parking position T,and calculates the guidance route R for placing the towed vehicle 12 andthe towing vehicle 10 in the parking space P. In this case, the same orsimilar effects as in the above embodiment are attainable.

Further, the above embodiment has described an example of moving thetowed vehicle 12 to the parking space P. However, the application of theperiphery monitoring device is not Limited to parking. It is applicableto moving the towed vehicle 12 closer to the side of the road or movingit for changing directions. That is, the periphery monitoring device canvisualize the dead area D occurring due to the motion of the towedvehicle 12 other than parking, and can attain the same or similareffects as the above embodiment.

Further, the above embodiment has described an example that the storingcontroller 62 stores the image data acquired by the image acquirer 50 bas it is as the parking target image. However, the image data may besubjected to image processing and stored in a form to be easilyassociated with the current image, such as the overhead image. The aboveembodiment has described an example that the parking target image storedby the storing controller 62 is displayed. if the target parkingposition T is in the guidance route R. In another embodiment, forexample, another window may be opened on the display device 26 aftersetting the target parking position T, to constantly display the parkingtarget image thereon.

A peripheral monitoring program executed by the CPU 36 a of the presentembodiment may be recorded and provided in an installable or executablefile format on a computer-readable recording medium such as a CD-ROM, aflexible disk (ED), a CD-R, or a digital versatile disk (DVD).

Further, the peripheral monitoring program may be stored on a computerconnected to a network such as the Internet and provided by beingdownloaded via the network. The peripheral monitoring program executedby the present embodiment may be provided or distributed via a networksuch as the Internet.

While certain embodiments have been described, these embodiments havebeen presented by way of example only, and are not intended to limit thescope of the inventions. Indeed, the novel methods and systems describedherein may be embodied in a variety of other forms; furthermore, variousomissions, substitutions and changes in the form of the methods andsystems described herein may be made without departing from the spiritof the inventions. The accompanying claims and their equivalents areintended to cover such forms or modifications as would fall within thescope and spirit of the inventions.

EXPLANATIONS OF LETTERS OR NUMERALS

-   10 Towing vehicle

-   12 Towed vehicle

-   24, 24 a, 24 b, 24 c, 24 d Imager

-   26 Display device

-   30 operational input

-   36 ECU

-   36 a CPU

-   50 Acquirer

-   50 a Peripheral monitoring-request acquirer

-   50 b image acquirer

-   50 c Coupling angle acquirer

-   50 d Vehicle position acquirer

-   50 e Guidance route acquirer

-   50 f Trailer-specification acquirer

-   50 g Index acquirer

-   52 Target setter

-   54 Determiner

-   54 a Coupling determiner

-   54 b Blind-spot determiner

-   56 Controller

-   62 Storing controller

-   64 Image controller

-   64 a Image-conversion controller

-   64 b Synthesis controller

-   64 c Comparison controller

-   66 a Trailer icon

-   66 b Vehicle icon

-   100 Periphery monitoring system

-   B Guidance reference point

-   C Target guidance point

-   D Dead area

-   P Parking space

-   R Guidance route

-   RV Actual image

-   T Target parking position

-   TV Overhead image

-   TV1 Complementary overhead image

-   

1. A periphery monitoring device, comprising: a coupling determiner that determines whether a towed vehicle is coupled to a towing vehicle to which the towed vehicle can be coupled; a target setter that sets a target moving position to be a target for moving at least the towed vehicle coupled to the towing vehicle; a storing controller that stores, as a moving target image, an image, including the target moving position, of a peripheral image generated by an imager provided at the towing vehicle; and an image controller that displays the stored moving target image in association with the towing vehicle or the towed vehicle included in a current image generated by the imager and currently displayed on a display device.
 2. The periphery monitoring device according to claim 1, wherein the storing controller stores at least the moving target image generated when the target moving position is set.
 3. The periphery monitoring device according to claim 1, wherein the storing controller starts storing the moving target image when the target moving position is set.
 4. The periphery monitoring device according to claim 1, further comprising: a blind-spot determiner that determines whether the target moving position enters a dead area caused by the towed vehicle in an imaging area of the imager, wherein the image controller displays the stored moving target image in association with the towing vehicle or the towed vehicle when the target moving position is to enter the dead area.
 5. The periphery monitoring device according to claim 4, wherein when the target moving position is in the dead area on the current image, the image controller superimposes at least an image of the target moving position included in the stored moving target image, on at least an area of the dead area, the area corresponding to the target moving position.
 6. The periphery monitoring device according to claim 5, wherein the image controller superimposes the moving target image on the dead area in a transparent mode.
 7. The periphery monitoring device according to claim 4, wherein when a first moving target image and a second moving target image exhibit a difference in content equal to or greater than a given value, the image controller displays the second moving target image in association with the current image, the first moving target image being stored when the target moving position is set, the second moving target image being stored immediately before the target moving position enters the dead area.
 8. The periphery monitoring device according to claim 1, further comprising: a position acquirer that acquires a current position of the towing vehicle with reference to a position of the towing vehicle at the time of setting the target moving position; an angle acquirer that acquires a coupling angle between the towing vehicle and the towed vehicle; and an index acquirer that acquires a trailer index corresponding to a size of the towed vehicle, the trailer index being superimposable on the current image, wherein in displaying the stored moving target image in association with the current image, the image controller determines a display posture of the trailer index in accordance with the current position of the towing vehicle and the coupling angle, and displays the trailer index on the current image in a superimposed manner. 